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ABSTRACT 

This paper describes the hardware and software systems approach used in the Automated Recertification 
Training System (ARTS), a Phase II Small Business Innovation Research (SBIR) project for NASA Kennedy Space 
Center (KSQ. The goal of this project is to optimize recertification training of technicians who process the Space 
Shuttle before launch by providing computer-based training courseware. The objectives of ARTS are to implement 
more effective CBT applications identified through a needs assessment process and to provide an enhanced 
courseware production system. The system’s capabilities are demonstrated by using five different pilot applications 
to convert existing classroom courses into interactive courseware. When the system is fully implemented at 
NASA/KSC, trainee job performance will improve and the cost of courseware development will be lower. 
Commercialization of the technology developed as part of this SBIR project is planned for Phase III. Anticipated 
spin-off products include custom courseware for technical skills training and courseware production software for use 
by corporate training organizations of aerospace and other industrial companies. 

INTRODUCTION 

Global competition and rapid changes in technology have increased the demand for employee education and job 
training. There arc three major reasons for this increase in demand [1], the first being that displaced workers need to 
be retrained. It is estimated that most workers will change jobs five or six times during their lives. Due to the 
dynamic nature of the U.S. economy, 1,5 million workers are permanently displaced each year and require 
assistance to reenter the workforce. By the year 2000, it is estimated that 5 to 15 million manufacturing jobs will 
require different skills than for today’s jobs, while an equal number of service jobs will become obsolete. 

Second, the work being performed at most companies is becoming increasingly complex. The use of computers 
and more sophisticated business processes require many employees to relearn how to perform their jobs. 
Furthermore, competition demands that there be constant change in the products and services that companies offer. 
This causes a ripple effect throughout the company in bringing these new products and services to customers. 

Changes in the organizational structure of companies also increase the demand for job training. One result of the 
difficult economic climate is that many companies are downsizing, increasing the need for cross training of workers. 
One of the principles of Total Quality Management (TQM) is the expansion of employee empowerment, with teams 
of employees performing a function or process with little or no direction from traditional management Companies 
that are turning to TQM principles are Finding that employees are unqualified for this empowerment without a large 
investment in training. 

The United States’ educational system fails to prepare many employees for the challenges of the modem 
workplace. The poor performance of U.S. high school graduates relative to their foreign counterparts on 
standardized tests is well publicized. Without supplementary training, the level of education of available American 
workers frequently fails to meet the requirements of employers. A recent study shows that one-fifth of displaced 
workers lack a high school education and that 20 to 40 percent of these workers are considered functionally illiterate. 

Due to changes in the economy and increases in skill level requirements for the workforce, many companies 
have expanded their employee training programs. Typical employee development programs at many large 
companies now include remedial training in reading, writing, and basic mathematics. Also, increased quality and 
safety requirements have caused companies to institute formal job or skill certification programs. However, the 
increased necessity of workforce training is costly in terms of time away from the job, travel costs to and from 
training sites, and expenses associated with classroom facilities, instructors* salaries, and administration. 
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Patients who have experienced discomfort on traditional computer keyboards have found relief when using 
THE VERTICAL . Further research with keyboard users showed individuals, and especially industry, will not 
accept a product which forces them to relearn the keyboard or takes extensive amounts of time to regain normal 
speed and accuracy. Based on the comments of hundreds of people who have interfaced with “THE VERTICAL” 
prototype, an average touch typist will regain speed and accuracy within hours. 


COMPETITION 

THE VERTICAL will compete with other companies which develop and market ergonomic computer 
keyboards, some of which are of greater size and may have greater financial resources. Ergonomic-Interface Keyboard 
System Inc.’s major competitors are the manufacturers of new “ergonomic” keyboard styles. However, those new 
keyboard styles which allow the user to pronate their hands and wrists during keystroking, maintain a “flat” keyboard 
similar to the major manufacturers, thereby incurring the same potential problems to the user. Others stray from the 
standard QWERTY key configuration and eliminate traditional typing formats altogether. They insist the user learn a 
new operational language in order to operate the board. Since, over the years, other unconventional keyboards such 
as the Dvorak System have been introduced yet have not been accepted by industry due to the need of retraining the 
user, it is felt by industry experts any keyboard which strays from the QWERTY format will not be accepted by the 
masses. One of the competitors states the optimal, ergonomic keyboard operates with the user’s hands in a 
vertically oriented position, yet their keyboard does not offer the user that possibility without eliminating the visual 
connection from the user to the board. Existing major computer keyboard manufacturers and major computer 
companies which manufacture their own keyboards are not considered competition due to existing lawsuits against 
them stemming from their ill-designed “flat” or “conventional” keyboards. 


Competition exists between the Company and some ergonomic experts who ignore the physical problems 
which the computer keyboard has been accused of creating. Their opinion is with proper body positioning and the 
correct use of properly designed ergonomic furniture, the problems of RMI and CTD could be eliminated. This is 
not the opinion of most medically and/or orthopaedically trained physicians. It does not address the damage 
keystroking plays on the user s body while the hands and wrists are pronated and arms are projected forward from the 
shoulders. Body position and furniture type alone cannot correct RMI and CTD problems. However, the Company 
does agree proper body positioning and the correct use of properly designed ergonomic furniture should be utilized 
while using “THE VERTICAL” so as to enhance the value of the keyboard’s design. 


Some experimentation has begun with hand written data entry, but this has limited application at best (for 

limited use by physically disabled operators, etc.). The error factor is also a serious problem with hand written data 
entry. 


FUTURE PRODUCTS 

The long term goals of The Company are towards the modification of other data input devices to obtain 
benefits similar to those of ‘THE VERTICAL”. Adaption of the basic design to meet specialized keyboard styles 
for governmental and research usage is expected to be an easy transition. Another logical extension of this design, 
which is now being worked on by The Company, is a vertically oriented Stenograph device. 
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Mrans must be found to make employee training more cost effective. Computer-based training (CBT) has been 
available for many years as either an alternative or a supplement to classroom training. Government-sponsored 
research and companies’ experience have shown CBT has the following potential benefits over conventional 
instructor-led training. 

• Training can be delivered and administered at lower cost. 

This is possible because a computer can be placed at or near the work site at a time that is convenient to the 
learner. Traditional classroom instruction settings and instructors are unable to provide the level of flexibility in 
location and time that a computer is able to. CBT can be delivered close to the woric site and can be scheduled at a 
time that is more convenient to the learner. Delivery of training on the computer also makes the course content more 
consistent and maintainable. 

• Training can be accomplished in less time. 

CBT allows individualization of lessons, whereby students access only the information that they need to learn. 
The modem ‘point-and-click user interface commonly used in CBT gives the learner increased control of the 
lesson and allows the training to be self-paced. Thus, faster learners are able to complete lessons in less time. 


Computer-based training improves learning when the training program is effectively developed. 


By building the capability for increased interaction into the lesson, immediate feedback and remediation is 
available to students. It is now possible to integrate audio and video into CBT presentations that appeal to multiple 

senses and various learning styles of students. Therefore, learners are more highly motivated to complete the training 
program. 6 

• Training achievement can be measured and tracked more easily. 

CBT enables tests and assessments to be embedded into the course, making recordkeeping easier. Student 
performance data can be stored for later analysis to improve lessons. 

Despite the potential benefits of CBT, the technology has not yet fulfilled its promise to make significant 
improvements in employee education and job training. Most applications of CBT are used in the United States 
military for combat training of personnel. Many of the training programs created for the military used specialized 
lgh-end applications developed for proprietary computer hardware. CBT did not become widely used in industry 
because it was too expensive to develop and the need for sophisticated training was not justified. Within the past few 
years, however, advances in personal computer hardware and software have made CBT more capable and 
affordable. The remaining challenge lies in demonstrating that CBT can be a cost-effective method of employee 
training in commercial industry. 


PROJECT BACKGROUND 

In January 1991, ENSCO, Inc. in Melbourne, Florida was awarded a Small Business Innovation Research 
(SBIR) contract to provide the Kennedy Space Center (KSQ with enhancements to its computer-based training. The 
focus of our project was to develop courseware for recertification training of technicians who perform pre-launch 
processing of the Space Shuttle. Processing of the shuttle vehicles requires that shuttle technicians be certified in 
approximately 500 technical skills. These skills include the operation of specialized test equipment, as well as 
performance of various types of mechanical and electrical repairs to the shuttle vehicle and its major systems. KSC’s 
technical training program requires that most technicians be recertified annually by attending a classroom refresher 
course and taking an examination. With each technician holding approximately seven certifications, this process 
takes more than 6000 worker hours per month away from work schedule. The purpose of ENSCO’s SBIR project is 
to apply CBT technology to existing recertification training courses, the result being the more effective delivery of 
training in less time than with conventional classroom training. 

Investigations of the target population and the types of work performed produced a number of factors that 
support the benefits of CBT over instructor-led training for skill recertification. First, shuttle technicians have 
varying training needs that tend to favor self-paced learning. Training requirements differ for electrical technicians. 
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mechanical technicians, and quality control inspectors. Depending on the individual technician's job, a particular 
knowledge or skill may be performed at a varying time interval. This variance can result in a heavy “forgetting 
curve” of the skill for some technicians but in little or none for others. Evidence also indicates that, due to 
differences in their ages and reading comprehension levels, technicians use diverse learning techniques. 

Another major factor favoring CBT is the dynamic nature of the shuttle processing work schedule, which 
depends upon the particular operations and maintenance tasks performed on the shuttle vehicle after each mission. 
Coordinating technicians’ training requirements with the changing work schedule presents challenges to the existing 
certification program. There are a number of instances in which a job cannot be performed as scheduled because a 
technician cannot attend the recertification class. There are also occasions when too many “no shows” in the 
classroom cause the cancellation of a scheduled training course. 

In January 1991, ENSCO began the research and development (R&D) effort on the process to convert existing 
recertification courses to CBT. This project required a strong working relationship between ENSCO and the 
Lockheed Space Operations Company (LSOC), which was the prime contractor responsible for shuttle processing 
and implementation of the technician training program. ENSCO also enlisted the services of the University of 
Central Florida to consult on training systems analysis and design. 

During Phase I of the SBIR contract, an early prototype application was delivered on a stand-alone PC. This 
prototype demonstrated the feasibility of CBT for recertification of typical shuttle processing tasks and showed that 
CBT could be accepted by technicians. However, Phase I research also showed the development process for 
sophisticated CBT was complex, particularly where multimedia computer technology was required. It was also 
determined that CBT would eventually need to be delivered by way of a network to achieve the greatest usage cost 
savings. Therefore, improvements in both the development and distribution systems were needed for CBT to be 
produced and distributed on a larger scale. 

ENSCO’s goal during Phase II of the project was to apply recent commercially available technology to the CBT 
process, which would result in a more efficient and cost-effective medium for recertification training. Our technical 
approach consisted of two major objectives: 

1. To provide a process for selecting good applications for CBT and determining the content of the CBT 
programs. 

2. To make the system for developing and maintaining CBT programs less costly and easier to use. 

Each of these objectives was addressed to overcome weaknesses in the current CBT system. In the following 
sections, these objectives are discussed in more detail. 

NEEDS ASSESSMENT PROCESS 

The first objective was satisfied with the needs assessment process. Needs assessment is a term that has many 
meanings in training and human performance literature. [2] The definition that seems most applicable to this project 
is the analysis of the training situation for the purpose of defining the requirements of a CBT development project . 
The needs assessment is conducted prior to the design of the CBT program. Its purposes are to identify the training 
goals and objectives and to select the appropriate media with which to present the content. A thorough needs 
assessment ensures that the instructional objectives are congruent to the performance job or task. A needs 
assessment also provides the basis with which to make rational decisions about how to apply CBT in the training 
program. 

The needs assessment process applied by ENSCO can be described in two stages, as shown in Figure 1. The 
first stage is task analysis, which determines the training objectives - what the trainee needs to know or needs to do 
to perform a task. Prior to analysis, data are collected from the existing lesson plans, from interviews with subject 
mauer experts and, if necessary, from observations of the task being performed. These data are then analyzed and a 
list of the knowledge and skills needed to perform the task is compiled. It is also important to identify prerequisite 
skills and knowledge during the analysis to avoid expending effort on developing unnecessary training content. 
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The second stage of the needs assessment process, media selection, determines the most effective way to 
provide the training. The inputs to the media selection process are the target audience characteristics, the training 
objectives, and the environment in which the training program will be deployed. The actual selection process ranges 
from a simple checklist verifying that CBT is a viable alternative to a comprehensive model for determining the best 
media to fit a particular set of training requirements and constraints. Possible choices include a hypertext document, 
an interactive multimedia courseware program, a system simulation, or an intelligent tutoring system. 



Figure 1. Needs Assessment Process. 


The outcomes of the needs assessment are a data base of training objectives and a revised plan of instruction. 
The objectives data base usually has a hierarchical structure in which broad job performance goals, also known as 
terminal objectives, are divided into specific learning (or enabling) objectives. Building the objective hierarchy is a 
crucial step in CBT development, as it becomes the foundation for the courseware design. Each objective should be 
linked to storyboards for laying out the instructional content and to the testing items for verifying that the student has 
met each objective. The second outcome of the needs assessment is a plan of instruction. This plan should include 
the level of the media chosen to fit the target audience, the training environment, and the total cost estimated for the 


There are major benefits to having a needs assessment First the assessment generates information with which 
to justify that the selection of CBT will result in improved trainee job performance. Second, there is a reduction in 
course design and development time as a result of the assessment’s identification of specific performance-based 
objectives for the CBT program. This results in the training content being more relevant, increasing the probability 
that students will accept and use the program. 3 


CBT PRODUCTION SYSTEM 

Q"® reasons that CBT is not widely used is that good CBT courseware has high development costs. 
Producing effecuve CBT is a complex undertaking that requires a team whose members have both creative and 
technical skills. These skills include: 


• script writing 

• graphic arts 

• audiovisual production 

• subject matter expertise 

■ instructional design 

• computer programming 

• systems engineering 

• performance measurement and assessment 

Building interactive CBT programs that are instructionally sound and interesting to the student requires a 
structured, yet flexible, design approach. CBT should be designed and built according to guidelines for screen 
design, instructional strategies, and testing methods. These guidelines have been established by educational 
psychoiogists and human factors engineers. For this project, the detailed requirements and standards used in the 
CB r lessons were established during the needs assessment phase of the project. 


A CBT production system was created and was used by development team members to build courseware 
according to established standards and guidelines. The hardware and software components of the production system 
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were configured using commercially available computer technology. Three major technical concerns influenced the 
creation of the production system: 

1. The hardware and software needed to be compatible with those used at KSC. 

2. The most sophisticated courseware applications to be produced would require interactive videodisc and 
digital audio capability. 

3. Some members of the CBT development team who would be using the system lacked extensive 
programming expertise. 

Since most of the CBT at KSC uses the IBM PS/2 platform, we selected compatible CBT production system 
hardware. This hardware configuration consists of the following components, as shown in Figure 2: 


Eraseable Optical Drive 


Video Disc 
Player 



Video Cassette 
Recorder 



Figure 2. CBT Production System Hardware Configuration. 

Most of the effort in creating the production system was expended in building the software environment. The 
system needed the capability for allowing the production of sophisticated CBT programs, while providing ease of 
use for members of the development team who were not experienced programmers. The software approach taken by 
ENSCO used a commercial-off-the-shelf authoring package as a foundation or shell and to integrate customized 
CBT production tools within the shell. The software configuration can be described as a pyramid, which is 
illustrated in Figure 3. Using this system, courseware programs could be written using the capabilities of the 
authoring package, the operating system, and the other system software that allowed operation of peripheral devices 
such as the audio card and videodisc player. The authoring package, Asymetrix® ToolBook®* was chosen because 
of its compatibility with other CBT development efforts at KSC. This version of ToolBook® runs under Microsoft 
Windows™ and comes with multimedia libraries that contain software drivers for audio and video hardware devices. 

The major features of the production system environment are listed below. 

• Password-protected access for student, developer, and system administrator 

• Hierarchical structure of scripting programs for ease of maintenance 

• Standard logic flow for different modes of instruction and reference, ensuring all training objectives are 
achieved by the student 
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• Templates for screen construction that assume common user navigation and media controls 
High level language for creation of time-sequenced presentation of media on the screen 



MS-DOS® is a registered trademark of Microsoft Corporation. 

Windows™ is is a trademark of Microsoft Corporation. 

ToolBook® is a registered trademark of of Asymetrix Corporation. 

Figure 3. CBT Production Software Environment 

Test engine for construction of test questions with question-answer randomizing capabilities and feedback 

• Development tools for showing and hiding media objects and for debugging the programs 

The benefits of the production environment include quality and functionality consistent with instructional 
system design principles, reduction in programming expertise required, and provision of a facility for program reuse 
and maintenance. 


PROJECT RESULTS 

Since Phase II of the project began in May of 1992, the system and technical approaches have been evaluated 
with actual KSC training courses that were converted to CBT applications. These pilot applications were selected 
primarily to illustrate how CBT could be used for training over a wide range of technical subject matter. The 
applications were also evaluated using the needs assessment process to determine their appropriateness for CBT. 
Examples of the types of training applications for which CBT programs were developed included: 

• Operation of a helium mass spectrometer for leak testing shuttle components 

• Operation of a computer-controlled laser tool for measuring steps and gaps between Orbiter tiles 
Safety precautions to be taken to counteract the hazards involved with high-pressure systems 

In the remaining five months of Phase II, we plan to deploy and evaluate the pilot courses in various training 
environments with diverse target populations. We designed pilot courses to use different media at different levels of 
sophistication - from a hypermedia reference manual to interactive multimedia courses for recertification with built- 
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in proficiency testing. The delivery methods by which the pilot CBT applications will be evaluated include stand- 
alone PCs, on-line local area networks, and concurrent training applications. CBT applications will be evaluated for 
use as a means of automated recertification, as a classroom training aid, and as a reference tool for the workplace. 

Initial data gathered from pilot studies show that the new approach favorably influences the training process. 
Using self-paced CBT, we expect that students will be able to cover course content in 25 to 50 percent of the time it 
would take them to cover the same material in a traditional classroom setting. We also expect a significant reduction 
in CBT development time because less programming will be required. Finally, we have CBT design standards to 
follow, making courseware more consistent and maintainable. 

COMMERCIAL APPLICATIONS 

The plan for Phase III of this SBIR project is to commercialize the technology developed as part of this project. 
As explained earlier, the market for CBT training in government and industry is expected to expand because of 
changes in the economy and demographic trends in the workforce. Insufficient numbers of instructors available to 
meet increasing training demands, as well as rapid growth in computer technology, will increase CBT’s market 
potential by its capability to disseminate information and increase interactivity with students at a steadily decreasing 
cost 

Our primary targets for marketing CBT technology are training departments of small and mid-size companies 
that require custom training and skills certification courses. A secondary marketing opportunity is to produce 
software to augment major CBT authoring systems. 

ENSCO’s systems approach is to harness technology to make CBT a more efficient and effective training 
method. The primary ways to make this happen are to select appropriate CBT applications and to specify lesson 
content accurately through the needs assessment process. Another means by which to increase the value of CBT to 
commercial industry is to reduce development and maintenance costs through enhancements of the CBT production 
system. 
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